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ABSTRACT 

We  show  that  the  iterative  signals  generated  by  a  simple  recurrently  connected 
network  of  only  3  sigmoidal  nodes  become  stable,  periodic  or  chaotic,  depending 
on  the  magnitude  of  a  global  parameter  (s)  that  controls  the  steepness  of  the  node 
responses.  High  values  of  s  produce  chaotic  signals,  and  low  values  lead  to  stable 
signals,  i.e.  unchanging  from  cycle  to  cycle.  The  transition  from  stability,  through 
periodicity  to  chaos  is  shown  to  be  consistent  with  Feigenbaum’s  model  of  critical 
nonlinear  systems. 

For  large  networks  that  are  recurrently  connected,  the  stability  also  depends  crit¬ 
ically  on  the  node  connection  weights  and  biases.  When  these  are  fixed  randomly,  it 
is  possible  to  find  large  systems  that  also  show  a  transition  from  stability  to  chaos 
by  the  adjustment  of  s.  When  the  weights  and  biases  are  allowed  to  evolve  by  a 
modified  Hebbian  rule,  which  depends  on  the  internal  state  of  the  network,  we  find 
that  the  system  becomes  stable  or  chaotic,  again  depending  on  s,  but  does  not  appear 
to  exhibit  periodicity. 

We  have  developed  a  software  simulation  of  a  machine  whose  motion  in  two  di¬ 
mensions  is  controlled  entirely  by  the  iterative  activity  of  a  recurrently  connected 
neural  network.  The  activities  of  some  randomly  chosen  nodes  drive  motor  responses 
that  control  the  movement.  In  the  absence  of  external  input,  the  machine  can  be 
made  to  spiral  into  inactivity  or  to  follow  a  chaotic  trajectory.  If  the  simulated  en¬ 
vironment  contains  one  or  more  emitting  sources,  and  the  machine  carries  sensors 
whose  responses  are  fed  into  the  network,  the  motion  is  influenced  by  the  sensory 
input.  Under  a  Hebbian- type  learning  scheme,  we  show  that  the  machine  is  strongly 
attracted  to  an  emitting  source  when  s  is  not  in  the  chaotic  regime.  The  motion  of 
the  machine  in  the  region  of  a  source  is  characterized  by  cycles  of  err.’'tic  attraction 
and  repulsion,  and  is  highly  reminiscent  of  the  motion  of  a  moth  near  a  light. 

It  is  possible  that  recurrent  networks  may  be  used  to  develop  navigation  and 
obstacle  avoidance  capabilities  that  are  self-learned  in  an  autonomous  land  vehicle. 


Acoflsslon  For 

UTIC  dFAs-l 

DTI:'  :v<' 

n 

r;i'  .'i;v;r>0 

□ 

Bv 

- - - - - 

"V/.  ij  PbiUty  Codes 


jAva,' i  and/or 
Dlat  j  .dpocial 

DRES-SR-589  UNCLASSIFIED  |^^(|  j 


NON  CLASSIFlfe 


R^;suM^: 

Nous  montrons  que  les  signaux  iteratifs  produits  par  un  reseau  simple  a  connexions 
recurrentes  de  seulement  trois  noeuds  sigmoidaux  devlennent  stables,  periodiques  ou 
chaotiques,  selon  la  grandeur  d’un  parametre  global  (i)  de  stabilite  des  reponses  des  noeuds. 
Selon  que  la  valeur  de  s  est  elevee  ou  faible,  la  reponse  est  chaotique  ou  stable,  c’est-a-dire 
constante  d’un  cycle  k  I’autre.  La  transition  d’un  6tat  stable  ^  un  etat  o^riodique,  puis 
chaotique,  est  en  accord  avec  le  moddle  des  syst^mes  non  lin6aires  critiques  de  Feigenbaum. 

Dans  les  grands  reseaux  connexions  recurrentes,  la  stabilite  depend  aussi 
grandement  de  la  ponderation  et  de  la  polarisation  des  connexions  entre  les  noeuds. 
Lorsque  ces  parametres  sont  fixes  au  hasard,  il  est  possible  de  trouver  de  grands  systemes 
qui  comportent  aussi  une  transition  de  I’etat  stable  I’etat  chaotique  selon  la  valeur 
attribuee  a  s.  Lorsque  nous  laissons  la  ponderation  et  la  polarisation  evoluer  suivant  une 
r^gle  Hebbienne  modifiee,  qui  depend  de  I’etat  interne  du  reseau,  nous  trouvons  que  le 
systeme  devient  stable  ou  chaotique,  toujours  selon  la  valeur  de  s,  sans  presenter  aucune 
periodicite. 

Nous  avons  mis  au  point  une  simulation  par  logiciel  d’une  machine  dont  le 
mouvement  en  deux  dimensions  est  entidrement  commande  par  I’activite  iterative  d’un 
reseau  neuronal  k  connexions  recurrentes.  Les  activit6s  de  certains  noeuds  choisis  au  hasard 
determinent  les  reponses  qui  commandent  le  mouvement.  En  I’absence  d’intrants  externes, 
la  machine  peut  etre  amenee  d  spiraler  vers  un  etat  d’inactivite  ou  a  suivre  une  trajectoire 
chaotique.  Si  le  milieu  simule  renferme  une  ou  plusieurs  sources  emettrices  et  que  la 
machine  porte  des  capteurs  dont  les  reponses  sont  introduites  dans  le  reseau,  le  mouvement 
est  fonction  des  intrants  sensoriels.  A  I’aide  d’un  module  d’apprentissage  de  type  Hebbien, 
nous  montrons  que  la  machine  est  fortement  attiree  vers  une  source  emettrice  lorsque  s 
n’est  pas  en  regime  chaotique.  Le  mouvement  de  la  machine  au  voisinage  d’une  source  est 
caracterise  par  des  cycles  d’attraction  et  de  repulsion  erratiques,  et  rappelle  beaucoup  celui 
d’un  papillon  de  nuit  pr^s  d’une  lampe. 

II  est  possible  que  des  reseaux  rdcurrents  puissent  servir  a  mettre  au  point  des 
moyens  de  navigation  et  d’evitement  des  obstacles  qui  sont  auto-appris  dans  un  vehicule 
terrestre  autonome. 
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1.  Introduction 


Part  of  the  work  performed  by  The  Advanced  Guidance  Concepts  Group  at  DRES 
is  research  on  radical  methods  for  developing  machine  intelligence.  The  long-term  goal 
is  to  develop  navigation,  obstacle  avoidance  and  object  recognition  capabilities  in  an 
adaptive,  intelligent,  autonomous  land  vehicle. 

Over  the  last  ten  years,  artificial  neural  network  (ANN)  techniques  have  shown 
promise  in  the  areas  of  nonlinear  mapping,  prediction,  pattern  recognition  and  adap¬ 
tive  nonlinear  control.  Beised  on  the  performance  of  biological  systems,  iterative, 
recurrently  connected  networks  should  be  capable  of  controlling  the  motion  of  a  dy¬ 
namic  system  in  a  changing  environment. 

The  work  presented  here  is  the  beginning  of  a  bottom-up  approach  to  machine 
intelligence  based  on  the  self-organizing  properties  of  recurrently  connected  ANNs 
We  wish  to  investigate  the  development  and  characteristics  of  the  behaviour  of  a 
machine  that  is  controlled  by  a  recurrent  network  when  it  is  allowed  to  evolve  in  an 
environment  that  provides  input  to  the  network  through  sensors.  Output  from  the 
network  will  drive  motor  responses  of  the  machine.  No  human  criteria  will  influence 
the  behaviour  of  the  machine;  i.e.  any  learning  that  occurs  will  be  unsupervised. 

Our  ANNs  consist  of  many  identical  nonlinear  processing  nodes  that  are  randomly 
interconnected.  Each  node  sums  the  weighted  input  from  many  other  nodes,  adds  a 
constant  bias  (or  offset),  performs  a  nonlinear  transformation  (sigmoidal)  and  passes 
the  output  to  many  other  nodes.  These  systems  are  connected  recurrently,  i.e.  there 
is  continuous  feedback.  In  the  absence  of  changing  external  input,  these  are  closed, 
iterative,  nonlinear  systems. 

Iterative,  nonlinear  systems  are  known  to  be  capable  of  generating  periodic  and 
chaotic  signals. 

Consider  a  nonlinear  function  of  a  variable  x,  whose  shape  is  determined  by  a  pa¬ 
rameter  A;  i.e.  /(x;  A).  Feigenbaum  [1]  showed  that  for  a  large  class  of  such  functions, 
values  generated  iteratively  exhibit  a  periodicity  that  doubles  as  the  magnitude  of  A 
increases  past  well-defined  values.  Beyond  a  critical  value  of  A,  iterative  values  of  / 
become  aperiodic  and  are  said  to  be  chaotic. 
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Iterative  signal  generation  may  be  written 


(1.1) 


Thus  if  x,+i  =  Xi  the  period  is  1,  and  i,  is  a  fixed  point  of  /.  If  x,+2  =  the 
period  is  2,  if  X{^4  =  x,  the  period  is  4,  and  so  on.  If  is  the  value  of  A  for  which  the 
n-th  periodicity  of  x  first  occurs,  Feigenbaum  showed  that  A„  converges  geometrically 
in  n  to  A 00,  for  which  the  system  is  aperiodic.  He  defined 

«,  =  (1.2) 
''n+2  ~  ''n+1 

and  found  that  Sn  approaches  a  universal  constant 

lim  =  6  =  4.6692..  (1.3) 

n— >00 

for  all  nonlinear  systems  that  exhibit  period  doubling. 

A  recurrently  connected  network  of  nonlinear  processing  nodes  can  therefore  be 
expected  to  generate  periodic  and  chaotic  signals  under  certain  circumstances. 

Recurrent  artificial  neural  networks  are  often  referred  to  as  Hopfield  nets  because 
of  the  pioneering  work  done  by  J.J.  Hopfield.  His  original  model  [2]  employed  a 
completely  connected  set  of  simple  linear  threshold  elements  (LTEs)  as  nodes.  The 
output  of  each  node  was  either  on  or  off  (0  or  1),  depending  on  whether  or  not  the 
sum  of  the  weighted  input  signals  was  above  the  node  threshold.  The  signals  passing 
between  nodes  were  modified  by  a  matrix  of  weights,  which  were  iteratively  adjusted 
to  minimize  a  Lyapunov  function.  Hopfield’s  later  work  [3]  employed  sigmoidal  nodes. 
The  output  of  a  sigmoidal  node  is  a  continuous,  bounded,  nonlinear,  monotonically- 
increasing  function  of  the  summed  input.  For  both  types  of  network,  Hopfield  showed 
that  stable  network  states,  giving  a  constant  output,  were  possible. 

Bruck  [4]  proved  that  Hopfield  nets  with  LTEs  always  converge  to  stable  or  cyclic 
states  of  period  2  or  4,  if  the  matrix  of  connection  weights  between  the  nodes  is 
symmetric  or  antisymmetric.  He  termed  these  “the  only  interesting  cases” ,  but  also 
showed  that  for  an  arbitrary  weight  matrix  there  exists  a  network  of  order  n  that  has 
a  cycle  of  period  2"^^.  For  example,  with  30  LTE  nodes  there  exists  a  network  of 
period  1024.  Bruck  did  not  consider  sigmoidal  nodes. 

The  presence  and  role  of  periodicity  and  chaos  in  biological  systems  has  been 
discussed  by  Freeman  and  coworkers  [5]-[7],  and  a  simulation  of  biological  neurons  by 
Chay  [8]  also  showed  the  presence  of  complex  oscillations  and  chaos.  A  mathematical 
proof  of  the  existence  of  period-doubling  to  chaos  in  sigmoidal  neurons  was  also  given 
by  Wang  [9].  The  generation  of  chaotic  behaviour  in  neural  networks  has  been  widely 
discussed  [10]-[16]. 

In  the  first  part  of  this  work  (Chapter  2)  we  show  .hat  the  generation  of  stable, 
periodic  and  chaotic  signals  in  recurrent  ANNs  can  be  controlled  by  a  single  global 
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parameter  s,  which  determines  the  steepness  of  the  node  response  functions.  A  simple 
recurrent  network  of  only  3  sigmoidal  nodes  can  exhibit  cyclical  behaviour,  with 
period  doubling  observed  from  1,2,..  1024  as  s  increases.  For  higher  values  of  s, 
chaotic  behaviour  results.  From  the  values  of  s  at  which  each  period  doubling  occurs, 
we  obtain  an  estimate  (4.669)  of  Feigenbaum’s  universal  constant  6,  showing  that  the 
process  does  indeed  lead  to  a  chaotic  system.  A  similar  study  [16]  of  a  system  of  3 
sigmoidal  nodes  with  delay  in  the  signal  transmission,  showed  that  periodicity  and 
chaos  can  also  result  from  connection  weight  and  bias  modifications. 

Beyond  the  first  chaotic  region  in  our  3-node  system,  there  exist  smaller  regions  of 
s,  for  which  cyclic  behaviour  and  period  doubling  reoccur.  We  have  observed  regions 
of  periodicity  7,  14,  28,  56,..,  and  also  9,  18,  36,  72..,  and  8,  16,  32,  64..  For  even 
higher  values  of  s,  cycles  of  unpredictable  period  occur  (3,  6,  5,  13,  4..).  This  erratic 
behaviour  is  shown  to  be  due  to  near  discontinuities  that  arise  when  the  sigmoids 
become  almost  step  functions  (quasi-LTEs). 

We  also  show  that  the  stability  of  large  networks  with  fixed  weights  and  biases 
can  be  controlled  by  the  adjustment  of  the  steepness  parameter,  s.  For  dynamic 
networks  (Chapter  3),  in  which  the  weights  and  biases  are  allowed  to  evolve  by  rules 
that  depend  on  the  internal  state  of  the  network,  the  stablity  may  again  be  controlled. 
When  the  activity  of  several  nodes  is  used  to  define  a  trajectory  in  two-dimensional 
space,  we  find  that  in  the  absence  of  external  input,  the  trajectories  either  spiral  in 
to  a  fixed  point  or  define  bounded  chaotic  regions. 

Thus  we  understand  quite  well  the  behaviour  of  a  simulated  recurrent  neural 
network,  evolving  by  internal  rules,  that  may  be  driving  the  movement  of  a  machine 
when  it  receives  no  sensory  input.  In  the  second  part  of  Chapter  3  we  then  examine 
the  effect  of  input  to  the  network  from  sensors  that  constantly  receive  intensity  from 
a  simulated  emitting  source,  such  as  a  light.  The  intensity,  and  hence  the  input  to 
the  network,  varies  as  the  position  of  the  machine,  in  a  true  dynamic  simulation.  The 
trajectory  that  such  a  machine  will  follow  is  shown  to  be  dramatically  affected  by  the 
presence  of  sensory  input. 

Finally,  in  Chapter  4  we  discuss  the  implication  of  these  results  for  the  possi¬ 
ble  development  of  machine  intelligence,  and  capabilities  of  obstacle  avoidance  and 
navigation  in  an  autonomous  land  vehicle. 
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2.  Recurrent  Networks 


We  consider  networks  that  are  composed  of  highly  connected  processing  nodes, 
the  output  of  each  node  n  being  a  sigmoidal  function  /„  of  the  total  input  signal 
We  define 

fn{Xn]3,XQn)  =  [1+ exp(-s(ar„  -  Xon))]"^  (2.1) 

The  function  /„  is  parametric  in  a  steepness  factor  s  and  an  offset  Xon-  Figure 
2.1  shows  the  sigmoidal  curves  for  s  =  4.0  and  8.0,  with  an  offset  of  0.5.  The  offset 
plays  the  same  role  as  a  bias  or  threshold.  We  will  also  refer  to  the  node  output  as 
its  activity. 

The  total  input  signal  to  the  n-th  node  at  cycle  t  is 

=  E  (2.2) 

m:=l 

i.e.  it  is  the  weighted  sum  of  the  outputs  from  the  previous  cycle  of  all  the  N  nodes 
in  the  network.  If  the  network  is  not  completely  connected,  some  of  the  weights  will 
be  zero.  Also,  the  diagonal  elements  of  the  weight  matrix  W  will  be  zero  for  nodes 
that  do  not  have  direct  feedback. 

In  this  work,  we  set  W„„  =  0  for  all  n,  and  we  may  normalise  the  sum  of  the 
absolute  values  of  the  weights  at  each  node  in  order  to  limit  the  total  input;  i.e. 

(2-3) 


where  c  is  a  constant. 

In  principle,  the  steepness  parameter  s  could  be  different  for  each  node,  but  in 
this  work  the  same  value  of  s  is  applied  to  every  node. 

The  iterative  output  of  the  network  in  matrix  form  is 

=  [1  +  exp(-3(Wf<‘-^)  -  Xo))]-‘  (2.4) 

The  weights  and  offsets  may  themselves  change  from  cycle  to  cycle.  There  are 
many  weight  updating  schemes  in  the  literature,  most  of  which  attempt  to  generate 
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Sigmoid  Functions  with  s  =  4.0,  8.0,  and  Xq  =  0.5. 

stable  network  states  in  response  to  given  initial  conditions;  i.e.  the  vector  f  becomes 
fixed  after  a  number  of  cycles.  References  16  and  17  are  collections  that  contain 
many  of  the  important  papers  on  weight  updating  algorithms.  In  Section  2.1  on  3- 
node  networks  and  in  Section  2.2  on  multinode  systems,  the  weights  are  not  altered 
from  cycle  to  cycle.  However,  Chapter  3  considers  networks  in  which  the  weights  and 
biases  change  2is  the  node  activities  change. 

In  the  next  section  we  show  how  a  simple  3-node  system  can  be  made  to  generate 
periodic  and  chaotic  output. 


2.1  A  3- node  recurrent  network 

Consider  the  simple  recurrent  network  of  3  nodes  shown  in  Figure  2.2.  Initially, 
we  consider  the  function  y{x)  that  is  generated  by  the  group  of  3  nodes  in  response 
to  the  input  x.  The  output  y{x^*^)  actually  occurs  after  2  network  cycles;  i.e. 

S,(iW)  =  (2.5) 

However,  to  demonstrate  the  property  of  period-doubling  to  chaos  in  a  way  that 
can  be  easily  related  to  Feigenbaum’s  work  [1],  we  consider  the  iterative  generation 
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of  the  output  of  the  3-node  group  to  be 

xW  _  y(xO))  ^  •  (2.6) 

2.1.1  Generation  of  symmetric  peak  functions 

The  sigmoidal  curves  generated  by  nodes  1  and  2  of  Figure  2.2  can  be  combined 
at  node  3  in  such  a  way  that  j/(x)  is  a  symmetric  peak  function.  This  can  be  done  by 
setting  VU31  =  —1^32.  Since  nodes  1  and  2  are  not  connected,  the  weight  matrix  is 

f  0  0  1\ 

W  -  0  0  1  (2.7) 

0/ 

W  satisfies  the  normalization  condition  (Eq.  2.3),  and  the  node  outputs  are 

/„  =  [1  -I-  exp(-5(x  -  xo„))]“^  (n  =  1, 2)  (2.8) 

and 

/a  =  [1  -f  exp(-s(p(i)  -  xos))]"^  =  y(x)  (2.9) 

where  ^ 

P(^)  =  -  fi(x;s,xoi)]  (2.10) 

If  the  two  sigmoids  fi  and  fi  have  different  offsets,  p(x)  i«  a  function.  Figure 
2.3  shows  p(x)  for  several  values  of  the  steepness  parameter  (s  =  10,  20,  30),  with 
xoi  =  0.65  and  102  =  0-60. 
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p(x)  with  3  =  10,  20,  30,  and  Iqi  =  0«65, 102  =  0.60. 

The  function  y{x)  also  depends  parametrically  on  the  offset  for  node  3,  Xoa-  Figure 
2.4  shows  y{x)  with  X03  =  0.0,  and  using  the  s  ne  parameters  as  those  used  in  Figure 
2.3.  It  can  be  shown  that  the  asymptotic  values  of  y(x)  are  0.5,  and  that  as  s  increases 
the  maximum  approaches  1.0;  i.e.  for  103  =  0.0 

0.5  <  y(x)  <  1.0  (— 00  <  X  <  00)  (2.11) 

If  Xo3  is  not  zero,  the  baseline  of  y{x)  moves  up  (xo3  <  0)  or  down  (xqs  >  0)  the 
y-axis. 

2.1.2  Periodicity  and  chaos 

Figure  2.4  shows  the  interception  of  the  line  y  =  x  with  curves  of  y(x;  s)  for  3 
values  of  the  steepness  parameter  s. 

Feigenbaum  showed  [1]  that  when  the  magnitude  of  the  slope  of  y(x)  at  y  =  x  is 
less  than  1,  the  point  of  interception  is  a  stable  fixed  point  (xy);  i.e.  iterations  from 
all  initial  values  of  x  converge  to  xy.  Thus,  the  lowest  curve  in  Figure  2.4,  with  s  = 
10,  will  generate  iterative  values  of  about  0.65  for  x  and  y(x),  for  any  initial  value  of 
X.  This  can  be  seen  from  the  following  procedure: 

1.  choose  any  initial  value  on  the  x-axis;  e.g.  xi  =  0.2; 

2.  find  y{xi)  on  the  curve; 


3.  move  horizontally  to  the  line  y  =  x  to  find  xa; 
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i/(x)  with  s  =  10,  20,  30,  and  xqi  =  0.65,  Xo2  =  0.60,  xqs  =  0.0.  The 
line  1/  =  X  is  also  shown. 

4.  iterate  steps  2  and  3. 

The  iteration  converges  rapidly  to  the  interception  point  on  the  curve  with  s  =  10. 

It  is  possible  for  a  curve  to  have  more  than  ov.  stable  fixed  point.  For  example, 
consider  the  curve  of  Figure  2.5,  which  has  been  displaced  to  the  right  by  setting 
Xqi  =  0.8  and  Xo2  =  0.75,  and  hzis  s  =  15.0.  In  this  case  there  are  two  stable  fixed 
points,  near  0.54  and  0.79. 

In  Figure  2.5  there  is  also  an  unstable  fixed  point  near  0.64.  At  this  point  the 
slope  of  y{x)  is  >  1,  and  iterations  diverge  away  from  the  point  to  either  of  the  stable 
points.  Therefore  this  system  can  stabilize  to  either  of  two  values,  depending  on  the 
initial  value.  There  are  certain  regions  of  x  for  which  the  iterative  result  is  extremely 
sensitive  to  the  initial  value;  e.g.  near  0.64  and  0.92  in  Figure  2.5. 

When  many  sigmoidal  nodes  provide  input  to  another  node,  the  output  function 
can  contain  many  peaks.  Therefore,  large  Hopfield  networks  can  stabilize  to  any  of 
many  fixed  patterns,  and  which  pattern  results  may  be  very  sensitive  to  the  initial 
conditions. 

Consider  the  cases  in  Figure  2.4  where  |y'|  >  1  at  =  x;  i.e.  with  s  =  20  and  30. 
Now  the  iterations  diverge  away  from  p  =  x.  This  is  an  unstable  fixed  point.  As  s 
increases  from  10,  the  iterated  values  settle  into  a  cycle  of  period  2. 
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y{x)  with  2  stable  fixed  points;  s  =  15,  xoi  =  0.8, 102  =  0.75,  I03  = 
0.0.  There  is  also  an  unstable  point. 


This  period  doubling  occurs  because  the  curve  y{y(x)),  which  we  write  y^^^{x),  has 
2  stable  fixed  points.  An  example  is  shown  in  Figure  2.6,  for  s  =  15.0. 

As  3  increases,  the  two  stable  points  of  y^^^  become  unstable,  4  new  stable  points 
first  occur  for  y^^^  (Figure  2.7,  with  3  =  16.0),  then  8  for  (Figure  2.8,  s  =  16.5) 
and  so  on.  Period  doubling  occurs  ad  infinitum,  but  the  spacing  between  successive 
values  of  3  at  which  doubling  occurs  becomes  geometrically  smaller.  Thus  there  is  a 
rapid  convergence  to  a  value  Soo,  beyond  which  the  system  is  aperiodic,  or  chaotic. 

Table  I  lists  the  values  of  found  numerically,  at  which  the  n-th  period  doubling 
was  first  observed  to  occur.  Table  I  also  gives  the  values  of  Feigenbaum’s  constants 
(5„,  calculated  by  substituting  Sn  for  An  in  Eq.  1.2.  This  shows  a  rapid  convergence 
to  Feigenbaum’s  value  of  S  (4.669201..). 

It  is  quite  difficult  to  obtain  accurate  numerical  estimates  of  Sn,  for  several  reasons. 
First,  for  any  value  of  a  that  is  very  near  a  period  doubling  transition,  it  takes 
hundreds  of  thousands  of  iterations  of  the  x-values  to  stabilize  to  a  precise  fixed  cycle. 
For  example,  Figure  2.9  shows  the  slow  convergence  of  iterated  x-values  when  s  =  12.9, 
near  the  first  period  doubling.  As  3  approaches  Si  (12.9735..)  the  convergence  gets 
slower,  and  as  3  increases  further  a  stable  oscillator  of  period  2  eventually  emerges, 
as  shown  in  Figure  2.10  for  3  =  13.1. 

The  closer  a  is  to  a  period-doubling  transition,  the  greater  is  the  number  of  itera- 
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with  2  stable  fixed  points;  s  =  15,  xoi  =  0.65,  xo2  =  0.60,  xo3 
=  0.0.  There  is  also  an  unstable  point. 


Figure  2.7 

y^*^x)  with  4  stable  fixed  points;  s  =  16,  Xoi  =  0.65,  loa  =  0.60,  xqs 
=  0.0.  There  are  also  3  unstable  points. 
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Figure  2.8 

j/(®)(x)  with  8  stable  fixed  points;  3  =  16.5,  xqi  =  0  65,  xo2  =  0.60, 
xo3  =  0.0.  There  are  also  7  unstable  points. 


n 

period 

1 

12.9735 

2 

2.503 

2 

15.4103 

4 

4.124 

3 

16.3838 

8 

4.579 

4 

16.61984 

16 

4.646 

5 

16.6713832 

32 

4.665 

6 

16.6824763 

64 

4.669 

7 

16.6848542 

128 

- 

8 

16.68536348 

256 

- 

Table  I 

Values  of  s  at  which  period  doubling  first  occurs 
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0.6984 

0.6982 

0.6980 

X 

0.6978 

0.6976 

0.6974 

( 

Figure  2.9 

Slow  convergence  of  when  s  =  12.9,  near  the  first  period  doubling 
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Figure  2.10 

Convergence  to  a  cycle  of  period  2  =  13.1). 
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Figure  2.11 

Cycle  of  period  4  (s  =  16.0). 

tions  required  to  determine  the  stable  cycle,  and  so  some  criteria  must  be  chosen  to 
define  the  point  at  which  period  doubling  is  accepted  to  have  occurred.  Our  algorithm 
performs  the  following  two  tests  for  a  cycle  of  period  N‘. 

|xi-Xi_Ar|<A  (2.12) 

and 

ii,-x._Ml>e  (M<iV)  (2.13) 

We  used  A  =  l.OE-12  and  e  =  l.OE-08.  The  second  condition,  Eq.  2.13,  establishes 
the  uniqueness  of  at  least  one  other  member  of  a  cycle  of  period  N.  After  every 
iteration,  the  new  point  must  be  checked  against  up  to  N  previous  points.  Thus,  if 
for  example  the  period  =  1024,  thousands  of  checks  must  be  pcformed  on  hundreds 
of  thousands  of  iterations.  This  is  quite  CPU  intensive,  taking  :  .’veral  minutes  on  a 
typical  Unix  workstation.  We  use  a  Hewlett  Packard  9000/720.  All  calculations  were 
done  in  double  precision,  using  a  C  program. 

Figures  2.11  and  2.12  show  example  of  cycles  of  periods  4  and  8  respectively. 
They  show  that  when  the  value  of  s  is  not  near  a  transition,  a  stable  cycle  is  quickly 
established  after  only  a  few  iterations. 

Table  I  shows  values  of  s  that  generate  period  doublings  up  to  256.  ('ycles  of  period 
512  and  1024  were  also  observed  with  s  =  16.6854726  and  16.6854960  respectively, 
but  no  accurate  estimates  of  Sg  and  Sjo  were  attempted  numerically  because  the 
differences  are  approaching  the  limit  of  double  precision  accuracy. 
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Figure  2.12 

Cycle  of  period  8  (a  =  16.5). 

Table  I  also  indicates  that  a  region  of  chaotic  (aperiodic)  signal  generation  would 
be  expected  for  values  of  s  >  16.7,  and  in  fact  no  periodicity  was  observed  immediately 
beyond  this  value.  Figure  2.13  shows  an  example  of  chaotic  iterative  x  values  for  s  = 
17.0.  The  values  shown  are  an  arbitrary  set  of  200  consecutive  iterations. 

Iterative  periodicity  and  chaos  can  be  represented  as  trajectories  in  xy-sp&ce.  If 
we  take  each  input  signal  x,  and  its  output  yi  (i.e.  x,+i)  to  be  a  coordinate  pair, 
then  the  movement  of  these  points  represents  a  trajectory.  Figure  2.14  shows  the 
trajectory  for  the  cycle  of  period  8  that  was  shown  in  Figure  2.12.  Figure  2.14  is 
actually  a  plot  of  101  points  from  the  400th  to  500th  iterations.  This  shows  that  the 
system  is  constrained  to  move  in  a  closed  8-point  trajectory.  Each  of  the  8  points 
must,  of  course,  lie  on  the  curve  y(x).  Figure  2.15  is  the  trajectory  for  the  chaotic 
system  shown  in  Figure  2.13.  In  this  case,  the  trajectory  is  not  closed.  Plotting  more 
points  results  in  completely  filling  the  curve  j/(x)  to  which  the  trajectory  is  confined. 

As  3  increases,  the  system  remains  chaotic  until  a  cycle  of  period  7  appears  near 
s  =  20.0.  Period  doubling  then  reoccurs  as  s  increases,  as  summarized  in  Table  II. 
The  way  in  which  this  region  of  periodicity  emerges  from  a  chaotic  region  can  be 
understood  by  looking  at  the  curve  y^’^^(x)  for  s  =  20.0,  where  the  period  is  7.  Figure 
2.16  shows  that  this  curve  has  7  points  where  |j/'|  <  1,  where  the  line  y  =  x  intercepts 
maxima  or  minima.  Figure  2.17  shows  the  same  curve,  but  with  s  =  19.0,  just  before 
the  onset  of  periodicity. 
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Figure  2.13 

Chaotic  signal  generation  (s  =  17.0). 


Figure  2.14 

Trajectory  of  a  cycle  of  period  8  (s  =  16.5). 
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Figure  2.15 

Chaotic  trajectory  (s  =  17.0). 


n 

Sn 

period 

0 

19.996 

7 

1 

20.090 

14 

2 

20.131 

28 

3 

20.140 

56 

4 

20.1415 

112 

5 

20.1419 

224 

Table  II 

Approximate  values  of  s  in  second  period  doubling  region 


with  7  stable  fiX( 
a;o3  =  0.0. 


with  no  stable  fixed 

Xo3  =  0.0. 
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n 

Sn 

period 

0 

20.600 

9 

1 

20.602 

18 

2 

20.6032 

36 

3 

20.60351 

72 

4 

20.60358 

144 

0 

20.95 

8 

1 

21.00 

16 

2 

21.024 

32 

3 

21.029 

64 

4 

21.030 

128 

Table  III 

Approximate  values  of  s  in  3rd  emd  4th  period  doubling  regions 


The  second  region  of  periodicity  spans  a  much  smaller  range  of  s  than  the  first, 
and  at  s  =  20.15  the  system  is  again  chaotic.  However,  because  of  the  emergence 
of  new  maxima  and  minima  in  the  curves  further  regions  of  period  doubling 

occur.  At  s  =  20.60,  a  cycle  of  period  9  is  established,  period  doubling  reoccurs,  then 
again  at  s  =  20.95  a  cycle  of  period  8  appears.  These  3rd  and  4th  regions  of  period 
doubling  are  summarized  in  Table  III. 

Beyond  the  4th  period  doubling  region  the  system  is  chaotic  until  about  s  =  24, 
where  a  cycle  of  period  3  appears.  Now  the  sigmoids  are  so  steep  that  they  are 
almost  step  functions.  At  s  =  23.0  (Figure  2.18)  heis  no  value  for  which  |j/'|  < 

1  where  y  =  x,  but  when  s  =  25.0  (Figure  2.19)  there  are  three  stable  fixed  points. 
This  system  of  period  3  remains  stable  as  a  increases  from  24  to  47;  i.e.  no  period 
doubling  occurs.  This  is  due  to  the  flat  extrema  produced  by  the  quasi-step  functions. 

At  a  =  48,  a  cycle  of  period  6  occurs.  However,  this  is  not  a  process  of  period 
doubling  leading  to  chcios,  but  rather  a  series  of  abrupt  and  unpredictable  transitions. 
As  a  increases,  cycles  of  period  5,  13,  4, 6, 5  and  7  appear  for  a  =  50,  51,  52,  61,  62  and 
70  respectively.  Figures  2.20  and  2.21,  showing  y^^^{x)  for  a  =  47  and  49  respectively, 
demonstrate  the  origin  of  these  sharp  transitions,  arising  near  discontinuities  in  the 
steep  quasi-step  functions.  Figure  2.22  shows  the  6  fixed  points  for  a  =  49,  and 
Figures  2.23  to  2.25  are  examples  of  cycles  with  periods  5,  13  and  4.  In  this  domain 
of  a  the  fixed  points  jump  erratically  from  one  flat  region  of  a  j/  iterate  to  another. 
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Figure  2.18 

y(3)(x)  with  no  stable  fixed  points;  s  =  23.0,  loi  =  0.65,  Xqj 

Xo3  =  0.0. 


Figure  2.19 

with  3  stable  fixed  points;  s  ~  25.0,  xoi  =  0.65,  Xoa 

Xo3  =  0.0. 


UNCLASSIFIED 


21 


with  3  stable  fixed  points;  s  =  47.0,  Xoi  =  0.65,  102  =  0.60, 

X03  =  0.0. 


Figure  2.21 

with  no  stable  fixed  points;  s  =  49.0,  xoi  =  0.65,  X02  =  0.60, 

Xo3  =  0.0. 
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Figure  2.22 

y^^^{x)  with  6  stable  fixed  points;  3  =  49.0,  loi  =  0.65,  102  =  0.60, 

Xo3  =  0.0. 


Figure  2.23 

with  5  stable  fixed  points;  s  =  50.0,  zqi  =  0.65,  X02  =  0.60, 

^03  —  0.0. 
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Figure  2.24 

with  13  stable  fixed  points;  s  =  51.0,  Xqi  =  0.65,  Xo2  =  0 

X03  =  0.0. 


Figure  2.25 

with  4  stable  fixed  points;  s  =  60.0,  xoi  =  0.65,  Xo2  =  0 

Xo3  =  0.0. 
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2.2  Multinode  Networks 

Having  determined  that  in  a  very  simple  recurrent  network  the  generation  of  sta¬ 
ble,  periodic  or  chaotic  signals  can  be  controlled  by  a  single  global  parameter,  we 
now  determine  if  stability  can  be  controlled  similarly  in  large  ANNs.  The  simple 
mathematical  analysis  of  the  iterated  functions  that  was  possible  for  the  3-node  sys¬ 
tem  is  no  longer  possible  when  the  output  signal  from  any  given  node  is  fed  back 
through  many  other  nodes  whose  outputs  may  not  reach  the  original  node  until  many 
iterations  are  completed;  i.e.  the  signal  that  is  fed  back  is  strongly  coupled  to  many 
previous  iterations.  We  have  therefore  focussed  on  the  numerical  results  of  iteratively 
calculating  the  vector  of  node  activities  in  large  networks. 

First  we  look  at  nets  in  which  the  weights  and  offsets  are  fixed  at  some  random 
initial  values  (sections  2.2.1  and  2.2.2)  and  the  system  is  iterated  with  various  values 
of  the  node  steepness  parameter  s.  These  rigid  systems  do  not  have  adaptive  or 
learning  properties,  although  it  may  be  possible  to  use  them  to  store  and  recall 
different  patterns  depending  on  the  initial  input  vector,  and  their  stability  properties 
are  relevant  to  an  understanding  of  large  nets  with  plastic  parameters.  Section  2,2.1 
considers  completely  connected  ANNs. 

Biological  systems  do  not  have  completely  connected  sets  of  neurons.  It  would  be 
physically  impossible  to  connect  all  the  billions  of  neurons  in  a  mammal,  for  example. 
Similarly,  in  a  simulation  of  a  recurrently  connected  ANN,  computer  processing  speed 
and  storage  capacity  limit  the  number  of  connections  per  node.  In  section  2.2.2  we 
describe  the  stability  of  a  partially  connected  ANN  with  fixed  parameters. 

In  Chapter  3  we  consider  dynamic  recurrent  networks,  in  which  the  weights  and 
offsets  change  from  cycle  to  cycle,  depending  on  the  input  and  output  at  each  node. 
Chapter  3  concludes  by  describing  the  interaction  of  a  dynamic  ANN  with  a  simulated 
environment,  through  nodes  receiving  external  input  and  transmitting  motor  output. 


2.2.1  Completely  connected  nets  with  fixed  weights  and  offsets 

We  have  simulated  several  completely  connected  multinode  networks.  The  nodes 
had  no  self-feedback  (W,i„  =  0).  The  weights  for  each  node  were  chosen  randomly 
between  -1  and  -|-1,  and  then  normalized  (Equation  2.3),  with  the  normalization 
constant  generally  chosen  to  be  the  number  of  nodes  in  the  network.  The  only  purpose 
of  this  normalization  was  to  ensure  that  each  node  could  receive  roughly  the  same 
total  input  signal.  The  offsets,  contained  in  the  vector  Xq,  were  chosen  randomly 
between  0  and  1. 

The  state  at  time  i  of  a  recurrent  network  of  N  nodes  is  given  by  the  vector 
It  is  calculated  from  the  state  at  the  previous  cycle  using  Equation  2.4.  The  initial 
components  of  the  state  vector  for  each  network  were  chosen  randomly  between  0  and 
1. 
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When  a  recurrent  network  operates  iteratively,  periodicity  must  be  detected  by 
testing  all  the  components  of  the  state  vector  with  all  the  components  from  previous 
cycles  (cf.  Equations  2.12  and  2.13).  For  example,  for  a  100  node  network,  the  outputs 
from  all  the  100  nodes  must  be  identical  to  those  from  a  previous  cycle  for  the  system 
to  be  termed  periodic. 

From  the  work  described  in  the  previous  section  on  3-node  networks,  we  expected 
that  periodicity  and  chaos  should  occur  for  some  random  multinode  networks,  and 
that  it  should  be  possible  to  control  this  behaviour  by  adjusting  the  node  steepness 
parameter  s  for  the  whole  network. 

Initial  work  on  nets  containing  20  and  100  nodes  supports  this.  A  20-node  network 
showed  cycles  of  period  1,7,  14  as  3  increased  from  1.0  to  2.1.  However,  beyond  about 
2.194,  we  could  detect  no  periodicity.  The  20-node  system  then  appears  to  be  chaotic. 
This  aperiodicity  persists  until  s  reaches  about  6.0,  where  a  cycle  of  1  occurs.  For 
higher  values  of  s,  erratic  periodicity  occurs,  much  as  in  the  3-node  case,  with  cycles 
of  period  38,  1,  142,  1,  31,  33,  13,..  being  observed  in  the  region  s  =  10  to  20. 

A  particular  randomly  initiated  100-node  network  was  stable  for  s  <  0.69,  i.e.  it 
converged  to  a  cycle  of  period  1.  However,  above  this  value  we  detected  no  periodicity. 
The  transition  from  stable  behaviour  to  chaos  seems  to  be  much  more  abrupt  for  this 
larger  network.  This  is  not  to  say  that  all  randomly  initiated  networks  will  exhibit  a 
transition  to  chaos  when  s  is  increased.  In  fact,  most  of  these  networks  simply  iterate 
to  a  stable  state.  However,  those  that  do  exhibit  a  transition  to  chaos  can  all  be 
stabilized  by  reducing  s. 

2.2.2  Partially  connected  nets  with  fixed  weights  and  offsets 

Because  it  was  a  convenient  structure  for  the  subsequent  studies  described  in 
Chapter  3,  we  focussed  on  a  network  of  90  nodes,  each  with  input  connections  from 
10  other  different  nodes.  The  connect’o”';  rboo«»n  randomly  from  anywhere  in 
the  network.  Other  network  sizes  and  numbers  of  connections  have  also  been  studied, 
but  the  results  are  essentially  the  same;  i.e.  the  stability  can  be  controlled  by  varying 
s. 

The  connection  weights  in  the  90-node  system  were  chosen  randomly  between  0.1 
and  0.9,  and  the  sign  was  also  assigned  randomly.  The  offsets  were  chosen  randomly 
between  -1.0  and  1.0. 

The  results  were  quite  straightforward.  For  s  <  5.6  the  network  converged  to  a 
fixed  state,  and  there  was  a  sharp  transition  to  chaos  around  s  >  5.7.  We  observed  no 
region  of  period  doubling  in  this  system.  However,  other  randomly  initialized  systems 
may  show  periodicity.  Since  our  goal  is  to  develop  adaptive,  learning  machines, 
a  detailed  study  of  the  conditions  under  which  period  doubling  may  occur  in  the 
transition  to  chaos  was  not  performed. 
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3.  Dynamic  Recurrent  Networks 


A  dynamic  recurrent  network  is  one  in  which  the  weights  and  offsets  change  from 
cycle  to  cycle,  depending  on  the  input  and  output  at  each  node. 

The  dynamic  variation  of  connection  weights  and  offsets  is  similar  to  the  changing 
synaptic  strengths  and  thresholds  in  biological  neural  systems,  and  offers  the  possibil¬ 
ity  of  machines  that  are  capable  of  learning  from,  remembering,  and  adapting  to  their 
environment.  This  requires  some  dynamic  interaction  between  the  network  and  the 
environment  through  nodes  that  receive  sensory  input  and  output  nodes  that  drive 
motor  responses. 

The  structure  of  such  a  dynamic  recurrent  ANN  is  described  in  section  3.1,  to¬ 
gether  with  its  properties  in  the  absence  of  external  input.  Output  node  signals  are 
used  to  drive  a  simulated  machine  and  its  two-dimensional  trajectories  are  shown  to 
depend  on  the  global  node  steepness  parameter  s. 

In  section  3.2  we  describe  the  effect  of  sensory  input  on  the  movement  of  the 
simulated  machine. 

3.1  A  partially  connected  dynamic  recurrent  ANN 

The  structure  of  the  recurrent  ANN  described  in  this  section  was  intended  to 
provide  the  basis  for  studies  of  the  dynamic  interaction  of  an  ,A,NN  with  a  simulated 
environment  containing  an  emitting  source,  results  from  which  are  given  in  section 
3.2. 

Our  dynamic  recurrent  ANN  (Figure  3.1)  has  3  distinct  regions,  which  we  call 
input,  internal,  and  output  regions.  Sensors  only  have  connections  to  the  input 
region,  and  motor  nodes  are  only  connected  to  the  output  region.  .Nodes  in  all  3 
regions  are  partially  interconnected  (randomly),  so  that  the  processing  is  recurrent 
throughout  the  ANN.  To  summarize,  the  following  functions  are  performed  in  each 
region; 

1.  input  region:  signals  from  sensors  and  other  network  nodes  are  received, 
processed  and  passed  to  other  nodes  in  the  network  by  a  limited  number  of 
randomly  chosen  connections; 
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Figure  3.1 

A  90-node  dynamic  recurrent  ANN  with  three  processing  regions. 


2.  internal  region;  signals  are  received  from  and  passed  to  randomly  connected 
nodes;  the  nodes  in  this  region  have  no  direct  external  input  or  output;  and 

3.  output  region:  signals  are  received  from  and  passed  to  anywhere  in  the  net¬ 
work,  but  not  directly  from  the  input  sensors;  the  motor  nodes  are  random!)’ 
connected  to  a  limited  number  of  the  nodes  in  this  region. 

An  example  of  this  arrangement  is  shown  schematically  in  Figure  3.1.  In  each 
of  the  regions,  a  node  is  shown  with  5  of  its  possible  connections.  The  sensors  only 
provide  output  signals,  and  the  motor  nodes  only  receive  input.  The  recurrent  nodes 
may  be  connected  for  either  input  or  output,  or  both.  In  our  simulation,  it  is  only 
necessary  to  store  the  input  connections  for  each  node  since  the  activity  of  any  node 
can  be  calculated  from  its  input  at  each  cycle.  The  node  activities  are  stored  from 
the  previous  cycle  and  then  updated  after  all  the  new  activities  have  been  calcu¬ 
lated.  Each  node  is  initialized  to  have  the  same  number  of  randomly  chosen  input 
connections,  and  every  node  provides  output  to  at  least  2  other  nodes. 
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We  used  90  recurrent  nodes  in  all  the  simulations  described  subsequently.  The 
nodes  were  divided  into  30  per  region,  as  shown  in  Figure  3.1.  This  was  a  convenient 
number  for  reeisonable  CPU  times,  and  provided  enough  nodes  in  each  recurrent 
region  for  at  least  10  unique  input  connections  per  node  without  it  being  difficult  to 
randomly  assign  the  connections.  The  sensors  each  had  10  random  connections  to 
different  nodes  in  the  input  region,  and  each  motor  node  had  10  random  connections 
from  different  nodes  in  the  output  region.  The  connectivity  of  our  network  is  thus 
about  10%.  We  have  not  yet  studied  the  effect  of  changing  the  connectivity. 

In  this  section,  the  sensor  outputs  were  adl  fixed  at  zero.  The  weights  for  the 
connections  from  the  sensor  nodes  to  the  input  region  were  all  set  to  1.0.  This  is 
unimportant  here  because  the  sensor  outputs  were  always  zero.  The  form  of  the 
sensor  response  as  a  function  of  incident  intensity  is  given  in  section  3.2. 

The  weights  for  the  recurrent  node  connections  were  reindomly  initialized  between 
0.1  and  0.9,  and  the  signs  were  also  assigned  randomly.  The  offsets  were  randomly 
initialized  between  -1.0  and  1.0.  The  weights  and  offsets  of  the  recurrent  nodes  were 
changed  dynamically,  as  described  in  section  3.1.1,  after  every  iterative  calculation  of 
the  network  state;  i.e.  the  node  activity  vector. 

The  motor  nodes  were  connected  with  constant  weights  of  magnitude  1.0  to  ran¬ 
domly  chosen  recurrent  nodes  in  the  output  region.  The  sign  was  alternated  between 
the  random  connections;  i.e.  these  weights  were  (1.0,  -1.0,  1.0,  -l.O,..).  The  motor 
node  weights  did  not  change  during  the  simulation.  However,  their  offsets  did  change 
dynamically  (see  section  3.1.1). 


3.1.1  Dynamic  variation  of  the  weights  and  offsets 

For  a  recurrent  node  i  at  cycle  t,  which  received  an  input  from  the  jth  node 
and  generated  an  output  //,  the  change  in  the  weight  connecting  nodes  i  and  j  that 
we  have  used  is: 


W 

(3.1) 

This  is  a  modified  form  of  the  synaptic  strength  modification  rule  proposed  by 
Hebb  [19].  The  principal  effect  is  to  strengthen  connections  carrying  strong  input 
signals  that  contribute  to  strong  output  signals.  Because  our  weights  may  be  positive 
or  negative,  we  have  included  a  sign  term  to  ensure  that  both  types  can  increase 
in  magnitude,  and  we  include  a  decay  term  (7)  so  that  the  weight  magnitudes  are 
bounded.  The  parameter  a  controls  the  maximum  change  that  can  occur  at  each 
cycle;  i.e.  it  is  a  growth  rate. 


DRES-SR-589 


30 


MCLASSIFIED 


We  also  eillow  the  offsets  of  the  recurrent  and  motor  nodes  to  vary  dynamically  in 
the  folowing  way: 


dxoi  =  -  0.5)  (3.2) 

This  ensures  that  every  node  continually  adjusts  its  offset  so  that  the  node  output 
approaches  the  centre  of  its  range.  No  node  can  be  permanently  off  because  its  offset 
is  too  high,  or  permanently  on  because  its  offset  is  too  low.  This  technique  has  the 
following  consequence:  in  a  recurrent  system  that  is  approaching  stability  all  the  node 
outputs  will  approach  0.5.  However,  for  networks  that  receive  changing  sensory  input, 
or  are  operating  in  the  chaotic  regime,  the  offsets  change  continually. 

From  Eq.  3.1  it  can  be  seen  that  for  a  system  approaching  stability,  the  weight 
change  approaches  zero  and 


-  ~  (3.3) 

If,  for  example,  the  growth  rate  is  set  to  ten  times  the  decay  rate,  then  all  the 
recurrent  node  weights  will  approach  a  magnitude  of  2.5.  Reaisonable  values  of  the 
update  parameters  are: 

a  =  0.01; 

7  =  0.001;  and 

0  =  0.01. 

These  values  ensure  that  the  maximum  change  in  a  weight  is  0.01  and  in  and  offset 
it  is  0.005  at  any  iteration;  i.e.  normally  less  than  1%  change  for  weights  and  offsets. 


3.1.2  2D  movement  of  a  simulated  machine 

Four  motor  nodes  are  shown  in  Fig.  3.1.  Since  their  outputs  are  all  positive,  we 
used  the  output  of  two  of  them  to  determine  a  change  in  the  x-coordinate,  one  node 
output  giving  the  other  -Sx.  Similarly,  the  other  two  motor  nodes  determine 
the  change  in  y.  The  magnitude  of  the  {x,y)  steps  taken  at  each  cycle  may  be  scaled 
for  convenience.  Consequently,  the  units  on  the  trajectory  diagrams  shown  in  this 
Chapter  are  arbitrary. 

Now  we  look  at  the  trajectories  generated  for  different  values  of  the  recurrent 
node  steepness  parameter  Sr.  The  motor  nodes  are  also  sigmoidal  in  our  system,  and 
the  steepness  parameter  (s^,)  for  all  of  them  was  assigned  a  fixed  value  of  4.0.  For 
each  motor  node,  this  generates  a  roughly  linear  response  to  the  total  input  that  it 
receives,  at  least  in  the  region  around  the  offset. 

Figure  3.2  is  an  example  of  the  trajectory  of  a  system  that  is  converging.  To 
generate  this  curve,  the  initial  random  network  was  iterated  for  5,000  cycles  to  allow 
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Figure  3.2 

Convergent  trajectory  with  5^  =  0.4  and  no  sensory  input 


the  weight  and  offset  schemes  to  take  effect,  using  the  update  rate  parameters  given 
in  section  3.1.1.  Then  the  motion  of  the  system  was  plotted  from  the  (0,0)  coordinate 
position  for  a  further  10,000  iterations.  The  movement  along  the  curve  then  represents 
the  change  in  {x,y)  for  each  iteration.  The  updating  scheme  clearly  generates  a 
smooth  convergence. 

The  trajectories  spiral  to  fixed  points  for  low  values  of  and  become  chaotic  as 
Sr  increases.  As  in  the  case  of  large  systems  with  fixed  weights  and  offsets  (section 
2.2),  there  is  a  sharp  transition  from  systems  that  converge  to  those  that  become 
aperiodic.  In  our  system,  this  occured  between  Sr  =  0.49  and  0.50. 

At  first  sight,  the  trajectories  of  systems  that  have  Sr  >  0.5  appear  to  be  closed, 
even  though  the  node  activity  vector  was  determined  to  have  no  exact  periodicity. 
With  Sr  ~  0.5  for  example,  Fig.  3.3  shows  the  trajectory  generated  by  lOK  iterations 
after  an  initial  500K  iterations  had  been  performed  from  the  random  initial  state  to 
ensure  that  it  had  not  in  fact  converged  to  a  periodic  system.  Figure  3.3  certainly 
appears  to  be  a  periodic  orbit.  However,  when  any  section  of  the  trajectory  is  greatly 
magnified,  it  is  clear  that  the  path  never  exactly  repeats  itself.  It  is  chaotic.  Figure 
3.4  is  a  magnified  view  of  the  top  right  hand  corner  of  the  orbit  shown  in  Fig.  3.3.  If 
the  number  of  iterations  is  increased,  the  lines  in  Fig,  3.4  simply  become  more  closely 
spaced;  e.g.  the  same  region  of  the  trajectory  is  shown  in  Fig.  3.5  after  20K  iterations. 

As  Sr  increases,  the  quasi-periodic  trajectory  first  becomes  more  complex  (Fig. 
3.6),  then  less  confined  (Fig.  3.7),  and  finally  completely  unstructured  (Fig.  3.8). 
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Figure  3.3 

Quasi-periodic  trajectory  with  Sr  =  0.5  and  no  sensory  input 
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Figure  3.4 

Magnified  view  of  the  chaotic  trajectory  with  s^  =  0.5  after  10,000 
iterations 
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Figure  3.9 

Simulated  machine  in  a  2D  world  with  an  emitting  source 

3.2  Effect  of  Sensory  Input 


We  have  simulated  the  movement  in  two  dimensions  of  a  circular  machine  that 
is  controlled  by  the  recurrent  neural  network  shown  in  Fig.  3.1.  The  six  sensors  are 
equally  spaced  around  the  circular  machine,  which  is  confined  to  move  in  a  circular 
world  containing  an  emitting  source.  The  arrangement  is  shown  in  Fig.  3.9. 

The  dimensions  and  units  used  in  the  simulation  are  arbitrary  because  the  motor 
node  outputs  can  be  scaled  to  adjust  the  machine  step  size,  and  the  source  intensity 
can  be  varied  at  will.  We  normally  use  a  world  radius  of  10  and  a  machine  diameter 
of  1.  The  intensity  at  the  source  is  normally  in  the  range  1  to  10,  and  is  modelled 
to  decrease  as  the  inverse  of  the  distance  from  the  source.  The  angular  size  of  the 
sensor  and  the  angle  that  the  sensor  surface  makes  with  a  line  joining  the  source  to 
the  sensor  centre  are  both  included  in  a  realistic  estimate  of  the  intensity  at  each 
sensor.  Also  the  position  and  size  of  each  sensor  is  used  to  determine  which  sensors 
are  partially  or  totally  illuminated.  The  response  of  a  sensor  that  is  not  illuminated 
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Figure  3.10 

Sensor  response  as  a  function  of  incident  intensity 

at  all  is  zero. 

The  function  that  we  have  used  to  generate  the  sensor  response  to  incident  inten¬ 
sity  is  shown  in  Fig,  3.10.  It  has  a  roughly  linear  response  in  the  centre  of  its  dynamic 
range,  and  saturates  at  high  intensity.  The  equation  used  to  generate  this  response 
is: 


/(x)= 

^  1  +  i'g(x) 

(3.4) 

where 

g(x)  =  exp(-Six) 

(3.5) 

and 

1/  =  exp(s,Xo,) 

(3.6) 

Si  and  xoi  define  the  steepness  and  offset  respectively  of  the  curve  shown  in  Fig. 
3.10.  The  subscript  i  is  used  to  indicate  input  to  the  network,  s^  =  4.0  gives  a  roughly 
linear  response  near  xo«.  The  value  of  Xq,  was  chosen  to  give  /(0.5)  =  0.5. 

Thus,  as  the  machine  moves  about  the  world,  driven  by  the  activity  of  the  recurrent 
nodes,  the  sensor  responses  change  dynamically  and  are  fed  into  the  network.  This 
provides  a  continual  feedback  from  the  environment.  However,  it  should  be  noted 
that  the  sensor  input  to  the  network  can  be  discontinuous,  because  a  sensor  may 
suddenly  become  illuminated  or  hidden  from  the  source.  Also,  the  machine  cannot 
go  beyond  the  boundary  of  the  world.  If  the  network  output  indicates  a  step  out  of 
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the  world,  the  machine  goes  as  far  as  possible  and  stops.  It  may  wait  many  iterations 
before  the  network  generates  an  acceptable  step  within  the  world. 

The  procedure  for  studying  the  effect  of  sensory  input  on  the  trajectories  is  then 
quite  straightforward.  The  machine  is  initially  placed  at  some  point,  usually  the 
centre  of  the  world  with  coordinates  (0,0),  and  is  initialized  with  random  connections. 
The  weights  and  offsets  may  be  either  random  or  from  the  final  state  resulting  from 
previous  iterations.  The  final  state  of  the  network  is  always  stored  at  the  end  of  a 
trial  so  that  it  can  be  continued  in  the  same  or  a  different  environment .  The  source 
is  given  a  constant  intensity  /,  and  the  simulation  is  started.  Our  program  gives  a 
graphical  animation  of  the  machine,  source  and  world  shown  in  Fig.  3.9.  It  shows  the 
machine  moving  in  the  world. 

If  the  source  is  {I  =  0),  the  machine  moves  ^long  trajectories  such  as  those 
shown  in  section  3.1,  depending  on  the  value  of  $r.  If  Sr  <  0.49,  the  motion  eventually 
ceases  (Fig.  3.2)  because  the  network  stabilizes.  If  Sr  >  0.5,  the  motion  may  appear 
periodic  or  chaotic.  (Figs.  3.3  -  3.8). 

3.2.1  Effect  of  pciitive  input 

An  intense  source,  e.g.  with  /  =  5.0,  has  a  dramatic  effect  on  the  trajectory  of  the 
machine  driven  by  a  non-chaotic  network. 

To  ensure  that  the  sensor  signals  dominate  input  to  the  nodes  in  the  input  region 
of  the  recurrent  network,  we  fixed  the  weights  for  the  connections  from  the  sensors 
at  -1-10.0  for  the  trials  in  this  section.  Recall  that  with  our  updating  scheme,  the 
reurrent  node  weights  tend  to  approach  a  magnitude  of  2.5  (section  3.1.1). 

When  a  machine  with  a  stable  network,  i.e.  one  that  has  converged  in  the  absence 
of  a  source,  is  placed  at  (0,0)  and  the  simulation  is  started  with  the  source  on,  the 
machine  immediately  starts  moving,  first  toward  the  source,  then  away,  and  eventually 
it  is  strongly  attracted  to  the  source.  It  appears  to  attach  itself  to  the  source  for  many 
iterations,  but  then  moves  rapidly  away,  eventually  being  attracted  again. 

It  is  difficult  to  express  the  effect  of  this  motion  without  actually  seeing  the  ani¬ 
mation.  Figure  3.11  shows  lOK  iterations  of  the  trajectory  of  a  machine  with  Sr  =  0.4 
(cf.  Fig.  3.2).  The  source  can  be  seen  as  a  small  circle  at  (-5.6,  -5.6).  The  lines  show 
the  trajectory  of  the  machine  centre,  which  is  0.5  units  from  the  machine  edge.  From 
this  it  is  evident  that  the  machine  tends  to  hover  around  the  source,  but  the  periods 
when  it  is  actually  immobile  while  touching  the  source  cannot  be  deduced.  However, 
Fig.  3.12  shows  the  distance  of  the  machine  from  the  source  at  each  iteration,  and 
from  this  the  attachment  periods  can  be  seen.  There  are  also  long  periods  when  the 
machine  is  immobile  at  the  edge  of  the  world,  but  the  attractions  continue  to  occur. 
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Figure  3.13 

Trajectory  with  5,  =  1.0  and  source  intensity  =  5.0 

A  chaotic  network  is  also  affected  by  the  presence  of  the  source.  With  Sr  =  1.0, 
the  machine  is  also  initially  attracted  towards  the  source,  but  rapidly  moves  away. 
It  moves  much  more  freely  over  the  entire  world  (Fig.  3.13),  and  although  there  are 
repeated  periods  of  attraction,  the  times  spent  touching  the  source  are  brief  (Fig. 
3.14). 

The  motion  of  these  systems  is  very  much  like  that  of  a  moth  hovering  near  a 
light, 

3,2.2  Effect  of  negative  input 

A  negative  input  signal  may  be  obtained  by  reflecting  Fig.  3.10  about  the  i-axis. 
When  this  negative  signal  is  fed  into  the  recurrent  network,  the  resulting  trajectories 
are  quite  different. 

For  Sr  <  0.5  the  machine  now  tends  to  avoid  touching  the  source.  Figure  3.15 
shows  the  trajectory  with  s,  =  0.4,  and  a  negative  sensor  response,  again  with  /  = 
5.0.  Comparing  this  with  Fig.  3.11,  it  can  be  seen  that  the  machine  is  much  less  likely 
to  touch  the  source.  In  fact.  Fig.  3.15  shows  15K  iterations,  and  only  once,  after  lOK 
iterations,  did  the  machine  actually  touch  the  source. 

For  values  of  Sr  that  are  in  the  chaotic  regime,  the  machine  exhibits  no  particular 
attraction  to  the  source.  Figure  3.16,  showing  15K  iterations  with  Sr  =  1.0,  indicates 
that  the  machine  is  more  likely  to  be  farther  from  the  source  than  for  the  case  of 
positive  input  (cf.  Fig.  3.13) 
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Figure  3.14 

Distance  from  source  {sr  =  1.0,  /  =  5.0) 


Figure  3.15 

Trajectory  with  negative  input  (s,  =  0.4,  I  =  5.0) 


UNCLASSIFIED 


DRES-SR-589 


UNCLASSIFIED 


4 


Figure  3.16 

Trajectory  with  negative  input  (3^  =  1-0,  /  =  5.0) 

A  similar  effect  is  obtained  if  an  inverted  input  signal  is  used;  i.e.  if  the  signal  is 
always  positive  but  goes  to  1.0  as  the  intensity  approaches  zero  and  goes  to  zero  as 
the  intensity  gets  large.  In  this  case  the  machine  also  appears  to  avoid  the  source. 
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4.  Discussion  and  Conclusions 


Iterative  signals  generated  by  a  recurrent  network  of  sigmoidal  nodes  with  no 
external  input  can  converge  to  stable  or  periodic  states,  or  they  may  be  chaotic. 
In  Section  2.1  we  showed  that  the  approach  to  chaos  in  a  3-node  system  may  be 
achieved  by  the  increase  in  magnitude  of  a  single  network  parameter,  s,  which  controls 
the  steepness  of  the  node  responses.  This  is  consistent  with  Feigenbaum’s  model 
[1]  of  a  nonlinear  system  driven  by  a  single  parameter.  As  s  increases,  a  stable 
system  becomes  periodic,  and  then  exhibits  period  doubling  to  chaos.  Beyond  the 
first  chaotic  region,  other  regions  of  periodicity  and  chaos  occur,  and  because  of  the 
step-like  functions  generated  by  higher  $  values,  a  region  of  erratic  periodicity  follows. 

The  behaviour  of  large  recurrent  networks  with  fixed  weights  and  offsets  is  more 
difficult  to  analyse.  However,  from  the  results  given  in  Section  2.2,  it  seems  that  the 
node  response  parameter  s  can  be  used  to  ensure  stability.  Below  a  certain  value  of 
s,  a  multi-node  network  will  converge  to  a  stable  state.  For  some  choices  of  weights 
and  offsets,  large  networks  can  become  chaotic  as  s  increases,  and  the  onset  of  chaotic 
signal  generation  appears  to  be  much  more  abrupt. 

A  dynamic  recurrent  network  (DRN),  in  which  the  connection  weights  and  node 
offsets  change  as  a  function  of  the  node  activities,  has  the  same  property.  In  Section 
3.1  we  showed  that  a  DRN  with  no  external  input  can  be  used  to  define  the  trajectories 
of  a  simulated  machine,  and  that  as  s  increases  these  trajectories  first  spiral  to  a  fixed 
point,  then  become  quasi-periodic,  and  are  finally  completely  unstructured  (chaotic). 

When  a  DRN  is  exposed  to  sensory  input,  the  trajectories  of  the  simulated  machine 
are  dramatically  altered.  The  machine  shows  a  strong  response  to  an  emitting  source 
(Section  3.2).  Depending  on  the  sign  of  the  input  signal  from  the  sensors,  there  is 
evidence  of  attractive  or  repulsive  behaviour.  The  motion  of  the  machine,  as  seen  in 
a  graphical  animation,  is  reminiscent  of  a  moth  near  a  light,  with  periods  of  erratic 
attraction,  hovering  and  repulsion.  For  nonchaotic  values  of  s,  the  equations  that 
drive  the  system  could  stablilize  in  two  possible  positions.  One  is  where  the  machine 
is  far  from  the  source,  so  that  the  input  is  weak  and  eventually  becomes  low  and 
constant,  and  the  machine  becomes  inactive.  This  is  observed  when  s  is  low  and  the 
source  intensity  is  weak.  Here  the  sum  of  the  node  activities  is  a  minimum.  The 
other  possible  stable  position  is  where  the  machine  is  touching  the  source,  so  that  the 
input  is  also  essentially  constant,  and  the  sum  of  the  node  activities  is  a  maximum. 


DRES-SR-589 


UNCLASSIFIED 


44 


UNCLASSIFIED 


For  strong  sources,  the  equations  drive  the  system  to  maxima  in  the  sum  of  the 
node  activities,  but  these  configurations  are  evidently  unstable,  because  after  several 
network  iterations  the  machine  moves  away  from  the  source. 

The  DRN  that  we  have  constructed  is  essentially  a  responsive  system.  In  the 
absence  of  a  source,  it  may  be  made  to  make  what  looks  like  a  random  walk,  using  a 
high  value  for  s.  Thus  the  machine  can  be  made  to  search  its  environment.  When  a 
source  is  detected  by  an  input  signal  from  one  or  more  sensors,  the  value  of  s  may  be 
lowered  so  that  the  machine  is  attracted  to  the  source,  if  the  input  is  positive.  It  may 
also  be  possible  to  effect  repulsive  behaviour  by  using  a  negative  or  inverted  input. 

So  far,  our  simulated  machine  has  no  memory.  When  a  source  is  illuminated, 
the  system  takes  some  time  to  respond  to  it.  It  may  be  possible  to  construct  a 
network  that  retains  a  memory  of  the  weight  modifications  associated  with  each 
sensor  response,  so  that  when  a  sensor  generates  a  signal  the  machine  immediately 
responds.  There  are  many  possible  variations  to  the  network  architecture  that  must 
be  investigated;  e.g.  changing  the  growth  and  decay  rates  in  the  modification  rules; 
increasing  the  number  of  nodes  and  connections  per  node;  and  using  distinct  network 
regions  that  have  different  weight  and  offset  updating  schemes. 

Our  weight  modification  scheme  may  be  called  a  pre-post  modification  rule,  be¬ 
cause  it  uses  the  strengths  of  input  signals  and  the  resulting  output  signal  at  each 
node.  Another  possiblity  is  a  pre- associative  or  pre-modulatory  scheme  in  which  strong 
coincident  input  signals  to  a  node  may  strengthen  their  connection  weights.  Kandel 
and  Hawkins  [20]  summarize  the  modification  rules  that  are  suspected  to  occur  in 
biological  systems. 

It  may  also  be  possible  to  use  regions  of  a  network  to  recognize  and  respond  to 
stored  patterns.  Depending  on  the  exact  initial  state,  a  multinode  net  may  converge 
iteratively  to  one  of  many  possible  stable  states,  and  this  has  been  proposed  as  a 
mechanism  for  pattern  recognition.  It  should  be  possible  to  avoid  iterative  chaos  in 
these  regions  of  a  network  by  adjusting  the  local  node  steepness  parameter.  The 
pattern  generated  may  then  drive  a  motor  node  response  in  a  predetermined  way. 

Our  initial  results  on  the  behaviour  of  dynamic  recurrent  networks  that  are  ex¬ 
posed  to  sensory  input  show  promise  for  the  eventual  development  of  autonomous 
vehicles  that  respond  adaptively  to  their  environment.  By  constructing  complex  re¬ 
current  systems  with  sensory  input  and  motor  output,  it  is  possible  that  intelligent 
machine  behaviour  may  be  created,  with  no  human  criteria  determining  the  machine 
responses.  The  expectation  is  that  it  will  be  possible  to  define  systems  that  lead  to 
useful  machine  behaviour. 
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We  show  that  the  iterative  signals  generated  by  a  simple  recurrently  connected  network  of  only 
3  Sigmoidal  nodes  become  stable,  periodic  or  chaotic,  depending  on  the  magnitude  of  a  global  para- 
m.eter  (s)  that  controls  the  steepness  of  the  node  responses.  High  values  of  s  produce  chaotic  signals, 
and  low  values  lead  to  stable  signals,  i.e.  unchanging  from  cycle  to  cycle.  The  transition  from  stability, 
through  periodicity  to  chaos  is  shown  to  be  consistent  with  Feigenbaum's  model  of  critical  nonlinear  systems 

For  large  networks  that  are  recurrently  connected,  the  stability  also  depends  critically  on  the  node  con¬ 
nection  weights  and  biases.  When  these  are  fixed  randomly,  it  is  possible  to  find  large  systems  that  also  show  a 
transition  from  stability  to  chaos  by  the  adjustment  of  s.  When  the  weights  and  biases  are  allowed  to  evolve  by  a 
modified  Hebbian  rule,  which  depends  on  the  internal  state  of  the  network,  we  find  that  the  system  becomes 
stable  or  chaotic,  again  depending  on  s,  but  does  not  appear  to  exhibit  periodicity. 

We  have  developed  a  software  simulation  of  a  machine  whose  motion  in  two  dimensions  is  controlled 
entirely  by  tho  iterative  activity  of  a  recurrently  connected  neural  network.  The  activities  of  some  randomly 
chosen  nodes  drive  motor  responses  that  control  the  movement.  In  the  absence  of  external  input,  the  machine 
can  be  made  to  spiral  into  inactivity  or  to  follow  a  chaotic  trajectory.  If  the  simulated  environment  contains  one  or 
more  emitting  sources,  and  the  machine  carries  sensors  whose  responses  are  fed  into  the  network,  the  motion  is 
influenced  by  the  sensory  input.  Under  a  Hebbian-type  learning  scheme,  we  show  that  the  machine  is  strongly 
attracted  to  an  emitting  source  when  s  is  not  in  the  chaotic  regime  The  motion  of  the  machine  in  the  region  of  a 
source  is  characterized  by  cycles  of  erratic  attraction  and  repulsion,  and  is  highly  reminiscent  of  the  motion  of  a 
I  moth  near  a  iighf 

it  is  possible  that  recurrent  networks  may  be  used  to  develop  navigation  and  obstacle  avoidance  capabil¬ 
ities  ihdi  aie  self-iearneu  in  cin  autonomous  (and  vehicle. 
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